United States Patent and Trademark Ofhce 



UNITED STATES DEPARTMENT OF COMMERCE 
United States Patent and Trademark OtBce 

Address: COMMISSIONER FOR PATENTS 



APPLICATION NO. 



10/589,613 



FILING DATE 



08/16/2006 



FIRST NAMED INVENTOR 



Venura C. Mendis 



ATTORNEY DOCKET NO. CONFIRMATION NO. 



23117 7590 09/18/2008 

NKON & VANDERHYE, PC 

901 NORTH GLEBE ROAD, 1 ITH FLOOR 

ARLINGTON, VA 22203 



NUNEZ, JORDANY 



PAPER NUMBER 



DELIVERY MODE 



Please find below and/or attached an Office communication concerning this application or proceeding. 

The time period for reply, if any, is set in the attached communication. 



PTOL-90A (Rev. 04/07) 



KJttiVrXi nvrliyjts OUff Iff fcff Jr 


Application No. 

10/589,613 


Applicant(s) 

MENDIS ET AL. 


Examiner 

Jordany Nunez 


Art Unit 

2175 





- The MAILING DATE of this communication appears on the cover sheet with the correspondence address — 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS, 
WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
eamed patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1 )^ Responsive to communication(s) filed on 19 June 2008 . 
2a )^ This action is FINAL. 2b)n This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) ^ Clalm(s) 1-28 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) \Z\ Claim(s) is/are allowed. 

6) |EI Claim(s) 1-28 is/are rejected. 
/)□ Claim(s) is/are objected to. 

8) 0 Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) 0 The specification is objected to by the Examiner. 

10)0 The drawing(s) filed on is/are: a)^ accepted or b)^ objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held In abeyance. See 37 CFR 1.85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 
1 !)□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12)0 Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 
a)n All b)n Some * c)^ None of: 

1 .□ Certified copies of the priority documents have been received. 

2. n Certified copies of the priority documents have been received in Application No. . 

3. n Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 



Attach ment(s) 

1 ) □ Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-41 3) 

2) □ Notice of Draftspereon's Patent Drawing Review (PTO-948) Paper No(s)/IVIail Date. 

3) □ Information Disclosure Statement(s) (PTO-1449 or PTO/SB/08) 5) □ Notice of Informal Patent Application (PTO-1 52) 

Paper No(s)/Mail Date . 6) □ Other: . 



PTOL-T26'(Rev^7-05)"'°^ 



Office Action Summary 



Part of Paper No./Mail Date 20080910 



Application/Control Number: 10/589,613 
Art Unit: 2175 



Page 2 



DETAILED ACTION 
Claim Rejections - 35 USC § 102 
The following is a quotation of the appropriate paragraphs of 35 U.S. C. 102 that form the basis 
for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public use or on 
sale in this country, more than one year prior to the date of application for patent in the United States, 
(e) the invention was described in (1) an application for patent, published under section 122(b), by another filed 
in the United States before the invention by the applicant for patent or (2) a patent granted on an application for 
patent by another filed in the United States before the invention by the applicant for patent, except that an 
international application filed under the treaty defined in section 351(a) shall have the effects for purposes of 
this subsection of an application filed in the United States only if the international application designated the 
United States and was published under Article 21(2) of such treaty in the English language. 

Claims 1-21 and 23-28 are rejected under 35 U.S.C. 102(b) as being anticipated by 
Madrane (6573907). 

Re claim 1, Madrane discloses a data handling device comprising a display (column 2 
lines 1 -2 for example) for displaying representations of the media objects (see figure 17, 18, root 
images for example), data storage means (see column 20 lines 24-26 for example) for allocating 
metadata tags (see column 13 line 66 to column 14 line 1, information designating these objects 
as "extractable" for example) to the media objects, an input device (interface viewer unit for 
example) comprising means to allow a representation of a selected media object to be moved 
into a region of the display representing a selected set of metadata tags (see figurers 17-18 for 
example), and means for causing the selected set of tags to be applied to the media object (see 
column 13 line 66 to column 14 line 3 for example). 
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Re claim 2, Madrane discloses a data handling device, configured to allow a user to 
generate additional metadata tags (see colunm 15, lines 18-20, additional text for example) 
having new values, such that the media objects may be further categorized. 

Re claim 3, Madrane discloses a data handling device, configured to provide a view of 
media objects to which one or more of a predetermined plurality of metadata tags have been 
applied (see figure 43 for example). 

Re claim 4, Madrane discloses a data handling device, configured to provide a view of 
media objects to which each of a predetermined plurality of metadata tags have been apphed (see 
figure 35 for example). 

Re claim 5, Madrane discloses a data handling device, wherein means are provided to 
provide user control of the maximum number of metadata tag sets to be displayed (define how 
many, see column 7 line27 for example). 

Re claim 6, Madrane discloses a data handling device, in which representations of the 
media objects are capable of being moved between regions of the display area representing sets 
of metadata tags having pre-defined values (identify which are "hot objects", see column 7 lines 
30-31 for example). 

Re claim 7, Madrane discloses a data handling device, comprising means for removing a 
representation of a selected media object from one display area and adding it to a second area, 
thereby applying the metadata tag set associated with the second area to the selected media 
object in place of the set of metadata tags associated with the first area (see column 7 lines 30-33 
for example). 
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Re claim 8, Madrane discloses a data handling device wherein a representation of a 
media object selected from a display area associated with a first metadata tag set applied to the 
media object may remain there whilst a copy of the selected media object is placed in a second 
area, thereby applying the metadata tag set associated with the second area to the media object in 
addition to the set associated with the first area (see column 20 lines 45-50 for example). 

Re claim 9, Madrane discloses a data handling device, providing means for indicating the 
number of media objects associated with a given set of metadata tags (column 7 lines 25-3 1 for 
example). 

Re claim 1 0, Madrane discloses a data handling device, providing means for indicating 
the number of metadata tags associated with one or more media objects (define how 
many. . .what image information, see column 7 lines 25-37 for example). 

Re claim 11, Madrane discloses a data handling device, providing means for identifying 
media objects to which no metadata tags have been applied by providing a display area 
representing an empty set (identify which objects in the scene are "hot objects", see column 7 
lines 30-34 for example). 

Re claim 12, Madrane discloses a data handling device, providing means for selecting a 
subset of the media objects for allocating a given set of metadata tags ("hot objects" and what 
image information will be displayed, see column 7 lines 30-34 for example). 

Re claim 13, Madrane discloses a data handling device, providing means for making the 
size of the display area allocated to each set of metadata tags proportional to the number of 
media objects portrayed therein (see column 5 lines 34-40 for example). 
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Re claim 14, Madrane discloses a computer program or suite of computer programs for 
use with one or more computers to provide any of the apparatus as set out in claim 1 (see column 
2 line 10 for example). 

Re claim 15, Madrane discloses a method of organizing and storing media objects for 
subsequent retrieval, the media objects being represented in a display, wherein in which 
metadata tags are applied to the media objects by selecting an individual media object from the 
display, and causing a set of metadata tags to be applied to the selected media object by placing a 
representation of the selected media object in a region of the display selected to represent the set 
of tags to be applied (see column 7 lines 25-35 for example). 

Re claim 16, Madrane discloses a method, in which a user may generate additional 
metadata tags having new values, such that the media objects may be fiirther categorized (see 
column 15 lines 17-20 for example). 

Re claim 17, Madrane discloses a method, wherein a view is provided of media objects to 
which one or more of a predetermined plurality of metadata tags have been applied (see column 
15 lines 17-20, see figure 43 for example). 

Re claim 18, Madrane discloses a method, wherein a view is provided of media objects to 
which each of a predetermined plurality of metadata tags have been apphed (see figure 35 for 
example). 

Re claim 19, Madrane discloses a method, wherein provision is made to control the 
maximum number of categories to be displayed (define how many, see column 7 line 27 for 
example). 
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Re claim 20, Madrane discloses a method, in which representations of the media objects 
are moved between regions of the display area representing sets of metadata tags having pre- 
defined values (see column 7 lines 30-31 for example). 

Re claim 21, Madrane discloses a method, wherein a representation of a media object is 
selected from a first display area associated with a first metadata tag set, and a copy of the 
selected representation is placed in a second area whilst the original representation remains in the 
first area, thereby applying the metadata tag set associated with the second area to the media 
object, in addition to the set associated with the first area (see column 20 lines 45-50 for 
example). 

Re claim 23, Madrane discloses a method, wherein the number of media objects 
associated with a given set of metadata tags is indicated (see column 7 lines 25-31 for example). 

Re claim 24, Madrane discloses a method, wherein the number of metadata tags 
associated with one or more media objects is indicated (see column 7 lines 25-37 for example). 

Re claim 25, Madrane discloses a method, wherein media objects to which no metadata 
tags have been applied are identified by providing a display area representing an empty set (see 
coliimn 7 lines 25-37 for example). 

Re claim 26, Madrane discloses a method, wherein a subset of the media objects may be 
selected for allocation of a given set of metadata tags (see column 7 lines 25-35 for example). 

Re claim 27, Madrane discloses a method, wherein the size of the display area allocated 
to each set of metadata tags is proportional to the number of media objects portrayed therein (see 
column 5 lines 34-40 for example). 
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Re claim 28, Madrane discloses a computer program or suite of computer programs for 
use with one or more computers to provide the method of claim 15 (see column 2 line 10 for 
example). 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

A patent may not be obtained though the invention is not identically disclosed or described as set forth 
in section 102 of this title, if the differences between the subject matter sought to be patented and the 
prior art are such that the subject matter as a whole would have been obvious at the time the invention 
was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

This application currently names joint inventors. In considering patentability of the 

claims under 35 U.S.C. 103(a), the examiner presumes that the subject matter of the various 

claims was commonly owned at the time any inventions covered therein were made absent any 

evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out 

the inventor and invention dates of each claim that was not commonly owned at the time a later 

invention was made in order for the examiner to consider the applicability of 35 U.S.C. 103(c) 

and potential 35 U.S.C. 102(e), (f) or (g) prior art under 35 U.S.C. 103(a). 

Claim 22 is rejected under 35 U.S.C. 103(a) as being unpatentable over Madrane 

Re claim 22, Madrane substantially discloses a method as set forth in claim 20 above. 

Madrane does not explicitly disclose a representation of a selected media object may be removed 

from a first display area associated with one metadata tag set when added to a second display 

area, thereby applying the set of metadata tags associated with the second display area to the 

selected media item in place of the set of metadata tags associated with the firs display area. 

Madrane teaches of applying the set of metadata tags associated with the second display area to 
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the selected media item in place of the set of metadata tags associated with the firs display area 
when added to a second display area. Deleting fiinctions are well known. It would have been an 
obvious matter of design choice to have a representation of a selected media object may be 
removed from a first display area associated with one metadata tag set when added to a second 
display area, since such a modification would have involved the mere application of a known 
technique to a piece of prior art ready for improvement. Where a claimed improvement on a 
device or apparatus is no more than "the simple substitution of one known element for another or 
the mere application of a known technique to a piece of prior art ready for improvement," the 
claim is unpatentable under 35 U.S.C. 103(a). Ex Parte Smith, 83 USPQ.2d 1509,1518-19 
(BPAI, 2007) (citing KSR v. Teleflex, 127 S.Ct. 1727, 1740, 82 USPQ2d 1385, 1396 (2007)). 
Accordingly Applicant claims a combination that only unites old elements with no change in the 
respective fimctions of those old elements, and the combination of those elements yields 
predictable results; absent evidence that the modifications necessary to effect the combination of 
elements is uniquely challenging or difficult for one of ordinary skill in the art, the claim is 
unpatentable as obvious under 35 U.S.C. 103(a). Ex Parte Smith, 83 USPQ.2d at 1518-19 
(BPAI, 2007) (citing KSR, 127 S.Ct. at 1740, 82 USPQ2d atl396. 

Response to Arguments 

Applicant's arguments have been fiiUy considered but are not persuasive. Examiner 
reiterates that references to specific columns, figures or lines should not be limiting in any way. 
The entire reference provides disclosure related to the claimed invention. Applicant argues that: 
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1) The Examiner cites to a passage of Madrane bridging columns 13-14 together with 
Figures 17-18 which are described at columns 30 and 42, respectively. These passages are only 
remotely associated with each other. The cited passage at columns 13-14 describes the 
identification of objects that may be of interest in a set of images making up a movie sequence 
(for example an object moving across the field of view - see Fig 7A), such that they can be 
tracked across the sequence despite the fact that they do not appear in the same position in each 
image. Note that the designer of the interface, not the user, determines which objects are tagged 
in this way. Nowhere does this passage teach or suggest "allowing a representation of a selected 
media object to be moved into a region of the display representing a selected set of metadata 
tags" or "causing a set of metadata tags to be applied to the selected media object by placing a 
representation of the selected media object in a region of the display selected to represent the set 
of tags apphed" as required by the present claims (page 9, first paragraph) 

Examiner disagrees. 

Applicant conveniently ignores the teachings of Madrane at Figs. 17 and 18 and columns 
30 and 42. Madrane teaches (column 30, 1. 5-17, fig. 17) that a designer may define a bounding 
box for objects around sparse, non-contiguous frames and further (col. 42, 1. 27-51, fig. 18) 
linking said bounding boxes to IP addresses through a hierarchical wizard. Thus, Madrane 
clearly teaches allowing a representation of a selected media object (e.g., an object) to be moved 
into a region of the display (e.g., an object moves across frames until it reaches the frame 
including the bounding box) representing a selected set of metadata tags (e.g., a bounding box 
with an associated IP address surrounds the object) and causing a set of metadata tags (e.g., IP 
addresses associated with bounding box) to be appUed to the selected media object by placing a 
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representation of the selected media object (e.g., the bounded object moves across frames until it 
reaches the frame including the bounding box) in a region of the display selected to represent the 
set of tags applied (e.g., the bounding box surrounds the object). Also, note that Madrane the 
designer of the interface is interpreted as the user. 

2) Figure 17 depicts how the position of such a tagged image is tracked across a series of 
images. "Metadata (represented by a title, a description and a URL) can be attached at different 
levels: at object annotation level, at the level of each object path or at the level of each bounding 
box." See Madrane at column 30, lines 1 1-14. However, nowhere does this passage teach or 
suggest "allowing a representation of a selected media object to be moved into a region of the 
display representing a selected set of metadata tags" or "causing a set of metadata tags to be 
applied to the selected media object by placing a representation of the selected media object in a 
region of the display selected to represent the set of tags applied" as required by the present 
claims (page 9, last paragraph). 

Examiner disagrees. 

As explained above, and as acknowledged above by Applicant, Madrane describes the 
identification of objects that may be of interest in a set of images making up a movie sequence, 
such that they can be tracked across the sequence despite the fact that they do not appear in the 
same position in each image. Thus, it is clear that Madrane teaches allowing a representation of a 
selected media object (e.g., an object) to be moved into a region of the display (e.g., an object 
moves across frames until it reaches the frame including the bounding box) representing a 
selected set of metadata tags (e.g., a bounding box with an associated IP address surrounds the 
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object) and causing a set of metadata tags (e.g., IP addresses associated with bounding box) to be 
applied to the selected media object by placing a representation of the selected media object 
(e.g., the bounded object moves across frames until it reaches the frame including the bounding 
box) in a region of the display selected to represent the set of tags applied (e.g., the bounding box 
surrounds the object). 



3) Applicants' invention is concerned with the manipulation of data, not of images per-se. 
There is nothing in Madrane providing for media objects to be tagged differently according to 
the region of the display in which the user chooses to position them (using an input device). 
Indeed, Madrane teaches the exact opposite in that it is important to Madrane's system that the 
same object, located in different positions in different frames, is identified as one and the same. 
Therefore, claims 1-28 are believed to patentably define over Madrane (page 10, second 
paragraph). 

Examiner disagrees. 

Nowhere in the claims does Applicant claim's recite "providing for media objects to be 
tagged differently according to the region of the display in which the user chooses to position 
them." It is not proper to read into the claims arguments by Applicant. 
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Conclusion 

THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of 
the extension of time policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 

MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1 .136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Jordany Nunez whose telephone number is (571)272-2753. The 
examiner can normally be reached on Monday Through Thursday 9am-7 :30pm. 

If attempts to reach the examiner by telephone are unsuccessfiil, the examiner's 
supervisor, William Bashore can be reached on (571)272-4088. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained trom the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would 
like assistance from a USPTO Customer Service Representative or access to the automated 
information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

JN 

9/10/2008 



/WILLIAM L. BASHORE/ 

Supervisory Patent Examiner, Art Unit 2175 



